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Method and apparatus for segmenting date to create mixed raster content planes 



(54) 

(57) An improved technique tor compressing a color 
or gray scale pixel map representing a document using 
an MRC format includes a method of segmenting an 
original pixel map into twoplanes (1 2.1 6). andthen com- 
pressing the data or each plane in an efficient manner. 
The triage is segmented by separating the mage into 



two portions at the edges. One plane contains image 
data for the dark sides of the edges, whle image data 
tor the bripjit sides of the edges and the smooth portions 
of the image are placed on the other plane. This results 
in improved image compression ratios and enhanced 
image quality. 
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Description 

[0001] This invention relates generally to image 
processing and. more particularly, to techniques tor seg- 
menting, classifying and/or compressing the digital rep- 5 
resentalion of a document. 

[0002] Documents scanned at high resolutions re- 
quire very large amounts of storage space. Instead of 
being stored as is. the data is typically subjected to some 
form of data compression in order to reduce its volume, 
and thereby avoid the high costs associated with storing 
it. "Lossless - compression methods such as Lempel-Ziv 
Welch (LZW) do not perform particularly well on 
scanned pixel maps. While 'lossy* methods such as 
JPEG work fairly well on continuous-tone pixel maps, 
they do not work particularly well on the parts o1 the page 
that contain text. To optimize image data compression, 
techniques, which can recognfee the type of data being 
compressed, are needed. 

[0003] Known compression techniques are descrfoed 
in US-A-5778092. US-A-5251271. US-A-5O60980, US- 
A-5784175, US-A-530331 3 and US-A-5432870. 
[0004] In one embodiment, the present invention dis- 
closes a method of segmenting a pixel map represen- 
tation of a document which includes the steps of: acquir- 
ing a block of the digital image data, wherein the digital 
image data is composed of light intensity signals in dis- 
crete locations; designating a classification for the block 
and providing an indication about a context of the block; 
segmenting the light intensity signals in the block into 
an upper subset and a lower subset based upon the des- 
ignated classification; generating a selector set which 
tracks the light intensity segmentation; and separately 
compressing the digital image data contained in the up- 
per and lower subsets. 

[0005] In another embodiment the present invention 
discloses a method of classrf ying a block of digital image 
data into one of a plurafity of image data types, wherein 
the block of data is composed of light intensity signals 
in discrete locations : which includes: dividing the block 
into a bright region and a dark region; dividing a low pass 
filtered version of the block into a bright region and a 
dark region; calculating average light intensity values for 
each of the bright region, the dark region, the filtered 
bright region and the filtered dark region; and comparing 
a difference between the bright region and the dark re- 
gion average light intensity values to a filtered difference 
between the bright region and the dark region average 
filtered light intensity values; if the average light intensity 
difference and the average filtered light intensity differ- 
ence are approximately equal finding a range of values 
in which the difference value falls, and classifying the 
block based upon the value range; and rl the average 
light intensity difference and the average filtered light in- 
tensity difference are not approximately equal finding a 
range of values in which the filtered difference value falls 
and classifying the block based upon the filtered value 
range. 



[0006] Some examples of methods according to the 
present invention will now be described with reference 
to the accompanying drawings, in which:- 



Figure 1 illustrates a composite image and includes 
an example of how such an image may be decom- 
posed into three MRC image planes- an upper 
plane, a lower plane, and a selector plane; 
Figure 2 contains a detailed view of a pixel map and 
io the manner in which pixels are grouped to 1orm 
blocks; 

Figure 3 contains a flow chart which illustrates gen- 
erally, the steps performed to practice the invention; 
Figure 4 contains a detailed illustration of the man- 
is net in which blocks may be classified according to 
the present invention; 

Figure 5 contains a detailed illustration of the man- 
ner in which blocks may be segmented based upon 
their classification according to the present inven- 
lion; 

Figure 6 contains the details of one embodiment of 
the manner in which block variation can be meas- 
ured as required by the embodiment of the invention 
shown in Figure 4; 
2B Figure 7 contains the details of an embodiment of 
the invention describing classification of blocks 
based upon the block variation measurement pro- 
. vided in Figure 6; 

Figure 8 contains the details of an embodiment of 
30 the invention for which context may be updated 
based upon the block classification provided in Fig- 
ure 7; and, 

Figure 9 contains the details of another embodi- 
ment of the invention for updating context based up- 
35 on block classification as provided in Figure 7. 

[0007] The present invention is directed to a method 
and apparatus tor separately processing the various 
types of data contained in a composite image. While the 
*o invention will described in a Mixed Raster Content 
(MRC) technique, it may be adapted for use with other 
methods and apparatus* and is not therefore, limited to 
a MRC format. The technique described herein is suit- 
able for use in various devices required for storing or 
45 transmitting documents such as facsimile devices, im- 
age storage devices and the like, and processing of both 
color and grayscale black and white images are possi- 
ble. 

[0008] A pixel map is one in which each discrete So- 
so cation on the page contains a picture element or "pixel" 
that emits a light signal with a value that indicates the 
color or, in the case of gray scale documents, how light 
or dark the image is at that location. As those skilled in 
the an will appreciate, most pixel maps have values that 
ss are taken from a set of discrete, non-negative integers. 
[0009] For example, in a pixel map for a color docu- 
ment, individual separations are often represented as 
digital values, often in the range 0 to 255. where 0 rep- 
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resents no colorant (i.e. when CMYK sepa a,.ons are 
used) a the lowest value in the range when lummance- 
! oJoi seoarations are used, and 255 tepresents 
ZZZ^ns^ colorantorthe highest value in 

^ncT in a gray-scale pixel map this ty P .ca«ly trans- * 
atesTc Tpixe. values which range .rom 0. tor black, to 
Sb lor the whitest tone possible. The pixel maps of 
concern in the currently prelerred embod.ment of the 
fl? invention are representations ol "scanned" im- 
SSSs wU are crealed by digitizing « 
Sh, reflected otl ol physical med* using a digrta I scar, 
ner The term bitmap is used to mean a binary pixel map 
in which pixels can take one of two values. 1 or 0. 
£nT<n Turning now to the drawings for a more de- 
X description of the MFC format, pixel map 10 rep- « 
esenung a color or gray-scale document is preferably 
^composed into a three plane page tormat as indcated 
?n F^ure 1. Pixels on pixel map 10 are pre erably 
groX in blocKs 18 (best i.lustrated in Figure: 2).£aJ- 
few lor belter image processing eHiciency. The docu- 
L formal is typically comprised ol an upper plane 12. 
r^TpTne iTa^a ee.ec.or plane ,6. Upper p^ne 
12 and tower plane 14 contain pixels that describ .he 
i riflta wherein pxels in each block IB 

^rJcTe^edtased upon predefined criten. - 
f o, example, pixels that have values above a certain 
fhTe^oW may be placed on one plane, while those wrth 
, « SlTare equal to or below the threshold are 
V f ^onThe othe^ne. Selector plane 16 keeps track 
^"SSSS-I*-™* lOandmapsal. p«e.s *> 
to an exact spot on either upper plane 12 or tower plane 

100111 The upper and tower planes are stored at the 
s^mebitdepthan^ as 
^ 10 but possibly at reduced ,esolut»on. Selector « 

ifi created and stored as a bitmap. It is impor- 
£ » recogTe that while the terms "upper" and "tow- 
TaTe useo to describe the planes on which data re- 
sides H is not intended to limit the invention to any par- 
licular arrangement or configuration. 

Sl21 After processing, an three planes are corn- 
Ed using e method suitable tor the type of data re- 

stoing thefeon. For example, upper plane 12 and lower 

'plane 9 14 may be compressed ^J^^^Z - 

K occ inn lechniaue such as JPEG, while seiecior 

°?Z 16 TcSnp^ssed and stored using a lossless 
5£r2£E2« such asgzip or CC.TT-G4. 1, would 
S apparent to one o« skin in .he an to compress and 

1 « JoH i^e of the output document. For example, 

erlbry be usedfor seleCor plane 16. since the particular 
n,! Re L lorrna. used must be one of the approved 

fo^rtnlhe present invention digita. image data is 
Eabfy processed using a MBC technique such as 
OescTeJ above. Pixe. map 10 represents a scanned 



image composed of light intensity signals dispersed 
throughout the separation at discrete locations. Again, 
a light signal is emitted from each of these discrete lo- 
cations, relerred to as "picture elements." "pixels" or 
•pels " at an intensity level which indicates the magni- 
tude ol the light being reflected from the original image 
at the corresponding location in that separation. 
[0014] In typical MRC fashion, pixel map 10 must be 
partitioned into two planes 12 and 14. Figure 3«ontams 
Tschematic diagram, which outlines the overall process 
used to segment pixel map 10 into an upper plane 12 
and a lower plane 1 4 according to the present invention. 
Block 18 isacquiredas indicated in step2l0; and Is clas- 
sified as indicated m step 220. In the prelened embod- 
iment ol the invention, block 18 win initially be classified 
as either UNIFORM. SMOOTH. WEAK EDGE or 
EDGE, and its context - either. TEXT or PICTURE - will 
be provided. The block will then be reclassified as either 
SMOOTH or EDGE, depending upon the initial classlfi- 
caiion and the context. Next, pixels in block 18 are seg- 
mented - placed on either upper plane 12 or tower plane 
14 according to criteria that is most appropriate for the 
manner in which the block has been classified as indi- 
cated in step 230. This process is repeated for each 
block 18 in orignal pixel map 10 until the entire pixel 
map 10 has been processed: Upper plane 12. lower 
plane 1 4 and selector plane 16 are then separately com- 
pressed, using e technique that is most suitable lor the 
type ol data contained on each, as indicated in step 240. 
I001S] Turing now to Figure 4. generally speekrtg. 
classif ication of blocks 18 into one of the four categones 
in step 220 as described above is preferably completed 
in three steps. First, the variation of pixel values within 
the block is determined as indicated in step 310. Block 
variation is best determined by using statistical meas- 
ures which will be described m detail below with refer- 
ence to Figure 6. Blocks with large variations throughout 
are most likely to actually He along edges of the image, 
while those containing little variations probably lie m uni- 
lorm or at least smooth areas. Measuring the venations 
within the block allows an inttial classfficatwn to be as- 
signed to it as indicated in step 320. Next, image data 
within each block 18 is reviewed in detailto allow context 
inlormatton <i.e. whether the region is in the text or pic- 
ture region of the image) to be updated and any neces- 
sary block re-classHicattons to be performed as shown 
in step 330. The UNIFORM blocks are reclassified as 
SMOOTH, and the WEAK EDGE blocks are upgraded 
to EDGE in a TEXT context or reclassified as SMOOTH 
in a PICTURE context. A smoothed version 20 of the 
image is also provided by applying a low pass filler to 
the pixel map 10. Smoothed image 20 is used in con- 
junction with original image data to offer additional in- 
lormation during classification, and also provries un- 
screened data for halftone regions. 
[0016] Figure 5 contains details of the manner in 
which block 18 is segmented into two planes, as provid- 
ed in step 230 ol Figure 3. The measuremen. begins by 
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lire , determining at step 410 whether the Hod £e.ng 
processed has initially been classed as an EDGE m 
step 220. II so. the values v p ol each pixel ,n the block 
are first compared to a brightness thieshold value v 
herein pixels that have values equal to or above JJ. are 
Swed es "bright- pixels, while those with values below 
raT-dark- pixels. Segmenting EDGE blocks simply 
deludes pacing darK pixels on upper l^* 
cated in step 440. and placing bright prxels on lowe 
Sane 14 as indicated in step 450. It it is deternruned at 
s ep 4 0 that block 18 is not an EDGE, all pixels in the 
block are processed together, rather than on a p.xel by 
Pixefbasis P .Tegmenting ol SMOOTH (non-EDGE) p«- 
e occurs "as loUows: it bk*k 18 is inlhe, rnids. ofatfjort 
run of blocks that have been classrfted ^ SMOOTH, 
and lurther. all blocks in this short run are dark (v <*) - 
all data in the block is placed on upper plane 2. It the 
enUre block 18 is substantially smooth (Le. in a long run) 
or te br^t (in a short run ot bright pixels), al. date .n 
block 1 8 is placed on lower plane 1 4. 
mm Turning now to Figure 6. the details of one em- 
SoinL Z the invention wherein initial block classrfi- 
SSTvte block variation measurement may be eccom- 
ptSd as required by step 3t 0 (Figure 4) are now de- 
bribed. A threshold, t,. which allows the block to be di- 
vToed into two portions is first calculated as indicated* 
s To in the preferred embodiment of the .nventron. 
this threshold is obtained by performing a h'««°9™ 
analysis on the data in the block, but many standard 
Methods can be used to perform this analyse. For ex- 
ample, the value that maximizes between dances of 
The criteria being used for separation or provides for 
rnaxLm separation between the two portions of the 
SocTctn be selected. Those skilled in the art will rec- 
^eTat other methods ol choosing the ^.^resh- 
old are available and the invention is not l.mrled to the 
emb^iment. Block 18 is then thresholded into these 
SSrts by comparing the light intensity value of each 
p^euo ^selected threshold t,. as indicated ^ tep 
520 As belore. if the pixel value v p is less than the 
freehold, the pixel is referred to as dark. It v p .s greater 
than or equal tot,, the pixel is bright. 
00181 As stated earlier, a smooth version 20 of he 
mage is obtained by applying a low pass filter to .the 
LTgL ^nage data. Average values lor bright and dark 
I xels are then obtained tor both the onginal and 
smoothed si ol image data. Looking firs, at the bright 
p££ one value calculated will be v BPlxEL . the average 
va.ue lor al. of the bright pixels in original pixel map JO 
,v a t ) which are located in the area covered by block 
1B as indicated in step 540. Another value. Vbsmooth. 
the average value for all of the bright pixels in smoothed 
vers7oT 2 § of ,he image which are located in the area 
covered by block 18 will also be obtained as *hown in 
step 560. Dark values are calculated similarly. That .s. 
! the average value for all of the dark pixels m 

oriS'pixe. map 10 (v p < U which are located in .he 
alea covered by block .8 will be ob.ained as shown in 



step 550. and v 0SMOO7H . the average value for all ot the 
dark pixels in the smoothed version 20 of the image 
which are located in the area covered by block 18 will 
be obtained as in step 570. Once these average values 
s aie obtained, .he distances d and d, between brighter 
and darker averages lor pixel map 10 and smoothed im- 
age 20 respectively are calculated as indicated in step 
580. That is d= V BPlXEL - V OP , XEL , and d, = Vbsmooth * 

VnSMOOTH S " Ce ^ ' S W' 08 '* 8lmOSt X ° 1 

10 contone images, the ratio of d/0, may be used to detect 

halftones. . 
10019] Figure 7 contents a detailed illustration of step 
320, of Figure 4. the preferred embodnnent of a process 
tor initially classifying blocks 18. As shown, a relative 
is comparison between d and d. is obtained as indicated 
in step 61 0 in order to determine whether the block con- 
tains contone (d » d.) or halftone data. Btock 18 will in- 
itially be classified as one of lour types: UNIFORM. 
SMOOTH. WEAK EDGE or EDGE according to the 
20 magnilude ol the distance d or d,. Distence d is used to 
classify contone blocks, while distance d, is used for 
halftones. For contone data d. the value from pixel map 
10. is compared to value Xo as shown in step 620. 
[0020] It d is very low (i.e. d< Xo), all pixel values In 
ss the block are substantially the same and the block is 
classified as UNIFORM a. step 640. If there are some- 
what small differences in pixel values in .he block such 
that x 0 <d<x 1 as shown in step 622. the block is classified 
as SMOOTH, at slep 650. If there are fairly large differ- 
so ences in pixel values in the block and x,<d<x 2 at step 
624. the block will be classified as WEAK EDGE. If the 
differences in the block are very large and dbfc at step 
624. .he block will be classified as an EDGE at step 670. 
10021) It d/d, is not approximately equal to 1. d» is 
35 compared to threshold y 0 at step 630. It should be noted 
there that two different sets of thresholds are applied for 
halftones and contones. Thus, on most occasions. 

xn^o. xi'yv and Vy* Tne p ,ocess used to c,assW y 

halftone blocks is similar to that used lor contone data. 

40 Thus, if d 6 <y 0 at step 630 the block is classified as UNI- 
FORM at step 640. If y 0 <d 6 <yi " ste P 632 « ^ e btock ,s 
classified as SMOOTH, at step 650. It y,<d«<y2 as indi- 
cated in step 634. the block is classified as a WEAK 
EDGE at step 660. II rf^ at step 634, the block will be 

«5 classified as an edge at step 670. 

[0022] Relerring now .o Figures 8 and 9, the details 
lor updating the context ot the block will now be provid- 
ed. The context ol a block is useful when the average 
between the dark and bright areas ot the block is rela- 

50 tively high. When this is the case, .he block can classi- 
fied as an EDGE as long as its context is TEXT. The 
context is initially set equal to PICTURE. It is changed 
to TEXT it one ol two rules is satisfied: (1) the block be- 
ing processed is in a long run ot UNIFORM blocks and 

55 the average ot the dark pixel values m me block is great- 
er than a preset brightness threshold, or (2) the block 
has been classified as either UNIFORM. WEAK EDGE, 
or EDGE, one of the top. left or right neighboring blocks 
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has a context which has been set equal to TEXT and 
the difference between that neighboring block and the 
current block is smaller than a preset propagation 

threshold. . . . 4 . 

I0023] Turning first to Figure 8, determining whether 
block context should be changed according to the first 
rule requires finding a run of blocks that have been clas- 
sified as UNIFORM as indicated in step 704. Finding a 
run of UNIFORM blocks typically involves companng 
the number of consecutive UNIFORM blocks to a run 
length threshold t^ as indicated in step 706. The run 
length threshold sets the number ol consecutive blocks 
that must be classified as UNIFORM for a run to be es- 
tablished As also indicated in step 706, V DPIXEL . the av- 
erage value ol the dark pixels tor consecutive blocks is 
compared to the brightness threshold V A large number 
of consecutive UNIFORM blocks with high bnghtness 
levels usually indicates that the blocks contain large 
background page areas (i.e. large white areas), thereby 
indicating thai lexl is present. Thus, if the number of con- 
secutive UNIFORM blocks exceeds t^ and V DPIXEL > 
t 8 , the context tor the block is changed to TEXT as indi- 
cated in step 708. 

10024] If either the number of identified consecutive 
blocks is too small to establish a run or the blocks are 

dark (V DPIX EL c W- * e conteXt wiH remai ° 66t GqUal t0 
PICTURE. Whether additional runs are present in the 
block will be determined as indicated in step 710. and if 
so the process will be repeated as indicated in the illus- 
tration. _ ( . 
100251 Turning now to Figure 9. changing the context 
of a block to TEXT under the second rule first requires 
providing a propagation threshold V The propagation 
threshold defines the level of brightness that will indicate 
that the block covers blank page areas. Under the sec- 
ond rule the context win be changed from picture to text 
at step 908 it the block is not SMOOTH (i.e. is UNI- 
FROM and EDGE or a WEAK EDGE) as shown in step 
802 either its top. left or right neighbor has a text context 
as indicated in step 804 and v BDIF , the average differ- 
ence between bright pixels in the block and bright pixels 
in the neighbor text context block is less than t, as 
shown in step 806. Neighbor blocks are checked be- 
cause presumably blocks that contain text will be locat- 
ed next to other blocks that contain text. However, the 
brightness value of the block is compared to that of its 
neighbor to assure that this is the case. In other words, 
even if ihe block has a neighboring block wilh a text con- 
text, a large difference between the average brightness 
of block and its neighbor means that the block contain 
does not contain the large blank page areas that indicate 
the presence of text. 

10026] Again, the present invention is directed to seg- 
menting the data by first identifying blocks that contain 
the edges ol the image and then separating the blocks 
such that those which contain the smooth data and 
bright sides of the edges are placed on the lower plane 
and the dark sides ol the edges are placed on the upper 



plane. Once each ol the respective planes is generated, 
ordinary MRC processing continues. That is. each plane 
is compressed using an appropriate compression tech- 
nique. In the currently preferred ernbodiment. upper 

5 plane 12 and lower plane 14 are compressed using 
JPEG while the selector plane 16. is compressed uising 
a symbol based pattern matching technique such as 
CCITT Group IV or a method of classHying scanned 
symbols into equivalence classes such as that de- 

io scribed in US-A 5.778.095 to Davies issued July 7, 
199B, the contents of which are hereby incorporated by 
reference. The planes are then joined together and 
transmitted to an output device, such as a facsimile ma- 
chine or storage device. 

is 

Claims 



1. A method of segmenting digital image data tor 
mixed raster content processing, comprising: 

a) acquiring a block of the digital image data, 
wherein the digital image data is composed of 
light intensity signals in discrete locations; 

b) designating a classification tor said block 
and providing an indication about a context of 
said block; 

c) segmenting said light intensity signals in said 
block into an upper subset and a lower subset 
based upon said designated classification; 

d) generating a selector 6et which tracks said 
light intensity segmentation; and 

e) separately compressing the digital image da- 
ta contained in said upper and lower subsets. 

2. A method of segmenting digital image data as 
claimed in claim' 1, wherein said classification indi- 
cates that said block contains substantially smooth 
data and/or substantially edge data. 

3. A method of segmenting digital image data as 
claimed in claim 1 or claim 2, wherein said classifi- 
cation data designating step lurther comprises: 

a) measuring an amount ot light intensity signal 
variation throughout said block; 

b) assigning a classification to said block based 
upon said measured light intensity signal vari- 
ation; and 

c) updating said context indication for said 
block, and designating classification for said 
block based upon said updated context. 



4. A method of segmenting digital image data as 
55 claimed in any of the preceding claims, further com- 
prising: 

a) dividing a low pass filtered version of said 
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block into a bright region and a dark region; 

b) calculating average filtered light intensity val- 
ues 1or said bright region and for said-dark re- 
gion; and 

c) obtaining a diflerence in average filtered light s 
intensity values between said bright region and 
said dark region. 

A method of segmenting a block of digital image da- 
ta into an upper and lower subset, wherein the block io 
of data is composed of light intensity signals in dis- 
crete locations, comprising: 

a) determining whether the block is located on 
an edge in the digital image; 

b) if the block is on an edge, comparing a mag- 
nitude of each light intensity signal in the block 
to a brightness threshold and placing said sig- 
nal in the upper subset If said Eght intensity 
magnitude exceeds said brightness threshold *° 
or in the lower subset if said light intensity mag- 
nitude is less than said brightness threshold; 
and 

c) if the block is not located on an edge, placing 
the block in the upper subset if the block is in a 
group of blocks that have light intensity values 
which are indicative of smooth and dark image 
data, and otherwise placing the block in the low- 
er subset. 

30 

A method of classifying a block of digital image data 
into one of a plurality of image data types, wherein 
the block of data is composed of light intensity sig- 
nals in discrete locations, comprising: 

3S 

a) dividing the block into a bright region and a 
dark region; 

b) dividing a low pass filtered version of said 
block into a bright region and a dark region; 

c) calculating average light intensity values for *o 
each of said bright region, said dark region, 
said filtered bright region and said filtered dark 
region; and 

d) comparing a diflerence between said bright 
region and said dark region average light inten- *5 
sity values to a filtered difference between said 
bright region and said dark region average fil- 
tered lighl intensity values; 

e) if said average light intensity difference and 
said average filtered light intensity difference so 
are approximately equal finding a range of val^ 
ues in which said difference value falls, and 
classifying said block based upon said value 
range; and 

f) if said average light intensity difference and ss 
said average filtered light intensity diflerence 

are not approximately equal finding a range of 
values in which said filtered difference value 



falls and classifying said block based upon said 
filtered value range. 

7. A method according to any of claims 1 to 4. wherein 
blocks are classified by a method according to claim 
5 or claim 6. 
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